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ABSTRACT 

Embry-Riddle Aeronautical University - Worldwide serves more than 36,000 online students across the 
globe, many of whom are military and other non-traditional students, offering 34 undergraduate, graduate, 
and professional education/workforce certificate programs, presented both online and via blended 
delivery modes. The centralized model of online course production and management produces and 
maintains more than 200 high quality turnkey-style courses, including several award winners. Faculty 
members in partnership with an instructional design production team design worldwide courses, working 
together to ensure course goals and learning objectives are achieved. The more than 800 geographically 
dispersed faculty members are monitored and coached throughout the course delivery process by a quality 
management team. With a student satisfaction rate above 85%, how does Worldwide ensure that quality is 
pervasive at every stage in the distance learning process? In addition, how does Worldwide innovate and 
continue to ensure the quality of design and instruction remains our top priority? The article addresses 
quality assurance components of the distance learning model at Worldwide, including administration, 
course design, instructor professional development, and course delivery. The actors and processes 
employed to harness Web 2.0, mobile, and cloud technologies to facilitate distance learning 
administration, teaching, and learning are detailed. 
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I. A BRIEF HISTORY OF EMBRY-RIDDLE AND THE WORLDWIDE 

CAMPUS 

Embry-Riddle Aeronautical University was born from the Embry-Riddle Company, founded in 1925 in 
Ohio by John Riddle and T. Higbee Embry This initial partnership brought together a flying school and a 
cargo/air mail delivery business. After World War 11, the school continued to teach civilian and military 
students, but it was not until 1965 that the school moved to Daytona Beach, Florida, and established itself 
as a residential college [1]. By 1970, Embry-Riddle was a University accredited by the Southern 
Association of Colleges and Schools (SACS). 

The history of Worldwide started with Embry-Riddle reaching out to non-traditional students on military 
bases by creating a teaching site at military installation at Fort Rucker, Alabama in the 1970s. Additional 
teaching sites were opened throughout the United States and Europe [2]. A second residential campus at 
Prescott, Arizona was opened in 1978 [3]. Today, Embry-Riddle Aeronautical University retains the two 
residential campuses with a third location, Worldwide, whose headquarters are in Daytona Beach, but has 
local campuses in 150 plus locations throughout the United States, Canada, Europe, Middle East and Asia 
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[4]- 

For the purposes of this article, an examination of online courses offered through Worldwide will be 
conducted; therefore, it is important to understand the structure of this branch of the institution. While it 
began with offering classes at military bases, today Worldwide offers students the ability to take classes 
within five modalities: classroom, online, EagleVision (EV) classroom, EV Elome and blended. Online 
courses began in 1994 with Graduate classes. By 1999, Undergraduate courses were added. Elaving 
online courses allows for “maximum convenience and flexibility for students with busy schedules or 
living in remote locations [4].” In 2007, EagleVision (EV) classroom began, which “is a Web video- 
conferencing platform that connects classrooms at different locations around the globe together in one 
live, real-time virtual classroom [4].” This enabled campuses to avoid cancelling classes due to low 
enrollments, for they can connect two or more classrooms together and serve the students’ desires for 
specific classes. With the success of the EV classroom, EV Elome was unveiled in 2009. This modality 
offers the same sort of flexibility as online courses, but students “benefit from real-time interaction with 
faculty and other students” in a synchronous format [4]. 

Classes for the degree programs of Worldwide are offered in all five modalities. Bachelor degrees 
include: Aviation Business Administration, Aviation Maintenance, Professional Aeronautics, Engineering 
Science, Fire Science, Transportation and Technical Management, with specializations in Logistics, 
Management of Information Systems, Project Management, or Occupation Safety and Elealth [5]. Master 
degrees are offered in the following programs: Aeronautical Science, Business Administration in 
Aviation, Leadership, Logistics and Supply Chain Management, Management, Project Management, 
Occupational Safety Management and Systems Engineering. Worldwide also offers a joint Ph.D. degree 
program with the Daytona Beach campus for Aviation [4]. This mixture of synchronous and 
asynchronous learning within the curriculum allows the students to find classes that fit into their lifestyle. 
This is an important element because the students of Worldwide tend to be older, nontraditional students 
looking for an andragogical approach to learning. 

The student population is almost 58% military and 42% civilian [6]. Worldwide began at a military base 
and it holds strong with the military. In fact, there are over ninety military locations with a Worldwide 
presence. The majority of Worldwide students are married, 70%, and of these students, 66% have 
children [6]. These non-conventional students respond positively to technology rich courses, as 
Worldwide enrollment numbers have increased from 64.949 from 2001-2002 to 92,737 in 2010-2011 [6]. 
The majority of the Worldwide courses are taught by adjunct faculty; however, the full time faculty play a 
vital role in the organization. They are often course developers of online courses and play the lead faculty 
role informing the adjunct faculty about the delivery of the courses and aiding in the success of 
instruction of such courses. The Faculty Senate also assisted with the success of course instruction 
among the five modalities by ensuring that the faculty dedicate a portion of their teaching load to each 
modality and faculty committees have supported the creation and growth of courses. This support was 
critical to the success of each of the modalities. 

To keep such a diverse population of student, instructors and staff connected, Worldwide utilizes social 
media and technology. Twitter and Facebook pages for Worldwide keep growing in followers. Last 
January through March, Facebook experienced a 131% increase in fans, with a 7% average increase in 
fans per month and averaged 172 interactions per month [6]. Twitter also had larger numbers of followers 
with a 152% increase, 8% in average per month and 122 interactions per month [6]. Furthermore, 
Worldwide is the only one of the three ERAU campuses to have a mobile application for smart phones. 
The applications boasts a full directory of faculty and staff, information related to specific majors and 
updated news. The application won best of show at 2011 UPCEA Annual Marketing and Publication 
Competition in April 2011 [7]. Additionally, the application won the Silver Award at the UCDA 
conference in November 2011 [8]. 


20 


Journal of Asynchronous Learning Networks, Volume 16: Issue 5 



Large Scale Quality Engineering in Distance Learning Programs 


II. OVERVIEW OF GOALS AND QUESTIONS 

This article examines the quality assurance that Worldwide infuses into their academic and faculty 
development courses provided in an online format. To keep the online classes with consistent, high 
quality on the cutting edge, the Instructional Design and Development Department (1DD) uses templates 
based on practicing principles to create academic rigorous courses. In order to teach these courses, 
faculty must go through a set of development courses to prepare to teach in the Blackboard platform used 
by Worldwide. Best pedagogy for online classes is provided and instructors gain a sense of how the 
courses will flow by taking them as a student prior to teaching them. To assist the instructors maintain 
quality, they are assigned to an Online Faculty Quality Manager (OFQM), who provides various 
resources for prior to, during and after the term. Additionally, the OFQMs conduct reviews of the 
instructors to verify that they are meeting the expectations and obligations set forth by the institution to 
provide a quality education to the student. 

Flow does Worldwide ensure that quality is pervasive at every stage in the distance learning process? In 
addition, how does Worldwide innovate and continue to ensure the quality of design and instruction 
remains a top priority? The article addresses components of the distance learning model at Worldwide, 
including administration, course design, instructor professional development, and course delivery, all with 
a focus on how instructional quality is assured at every stage. 

III. A BRIEF REVIEW OF LITERATURE ON QUALITY ASSURANCE 

Quality assurance (QA) models are influenced by numerous situational and environmental factors, 
including accreditation, technology, and competitiveness, to name a few. QA appears to be a core value, 
and underlies many policy decisions in a higher education organization [9-10]. While support for and 
views with regards to how online learning should be administered, particularly with the course design 
component, are diverse, Meyer and Barefield have found one significant factor that rises above the others 
and pushes institutions to seek ways to increase quality: accreditation [11]. According to the Council for 
Higher Education Accreditation (CHEA), accreditation provides a means for institutions to self-regulate 
educational quality through self-examination and peer review using a system of standards and outcomes 
for public disclosure [12], CHEA goes further to define QA as a “planned and systematic review process 
of an institution or program to determine that acceptable standards of education, scholarship and 
infrastructure are being maintained and enhanced [13].” As is standard in accreditation guidelines, 
accountability, control, and improvement are commonly considered to be the main aims of QA [9] [14]. 
Any administrative model for online learning should have a comprehensive QA approach in four main 
areas in order to achieve accountability, control, and improvement: (1) instructor professional 
development, (2) instructional design and course development, (3) quality assurance itself, and (4) 
assessment [15]. For the instructional design element of online course production, these aims can be 
achieved through the establishment and application of minimum standards and continuous reviews to 
ensure quality throughout the course development process [16-17]. 

Peer evaluations, assessment and outcomes alignment, continuous improvement methods, student 
satisfaction, and performance indicators are all leading approaches to measuring QA [9] [18]. The 
development of QA models for online learning have begun to emerge, and in alignment with current 
theoretical trends, their development has evolved into non-linear, often cyclical structures, such as the 
process-oriented one proposed by Abdous [9], This model identifies QA tasks organized within the 
instructional design process structure with three phases: (1) planning and analysis; (2) design, prototype 
and production; and (3) post-production and delivery. QA markers include clearly delineating the 
instructional design process for all development team members during each phase. During the first phase 
(planning and analysis), the use of a flowchart for procedures, a timelines for the production process, and 
preparation of templates and QA checklists are considered appropriate actions. During Phase 2 (design, 
prototype, and production), reviews and QA checklists are used during development to assure adherence 
to standards. Finally, during Phase 3 (post-production and delivery), end-user feedback is collected and 
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results in making applicable updates the content. In another QA model for online courses, Mihai bases an 
E-Modules Lifecycle and QA Mechanisms model upon Abdous, but goes a step further by delineating the 
stakeholders most involved in each stage of the instructional design QA process [9, 19]. 

External organizations affiliated with distance learning in one form or another, such as membership 
associations, technology providers, and of course, accrediting agencies, often establish their own QA 
initiatives. Quality Matters (QM) was established specifically for this purpose. The QM rubric was 
developed using national standards of best practice, is firmly rooted in research literature, and integrates 
accepted instructional design principles [32], The QM rubric by which online courses and programs are 
evaluated addresses eight areas: (1) course overview and introduction, (2) learning objectives 
(competencies), (3) assessment and measurement, (4) instructional materials, (5) learner interaction and 
engagement, (6) course technology, (7) learner support, and (8) accessibility [20], Popular learning 
management system providers, such as Blackboard, also promote their vision of quality through 
evaluation tools such as the Exemplary Course Program rubric, which addresses: (1) goals and objectives, 
(2) content presentation, (3) learning engagement, (4) interaction and collaboration, (5) assessment 
expectations and design, (6) learner support, and (7) templates [21], 

IV. QUALITY ASSURANCE AND STANDARDS IN COURSE DESIGN 

AND DEVELOPMENT 

Many of the QM and Blackboard (among other evaluation providers) criteria are addressed via the 
standard template used in all Worldwide online courses, but care has been taken to ensure that all other 
criteria are covered during the individual course design phase of the process. The QM program provided 
much of the baseline for the development of our standardized template and course design standards 
during the development and implementation of the “Gold” template in 2007, described in detail further 
below. As part of the review process for QM, the 1DD department submitted two courses: WEAX 201 
Meteorology 1, receiving 79 out of 80 points, and ENGL 221 Technical Report Writing, receiving 80 out 
of 80 points. The results were excellent (either perfect or near perfect scores were achieved), so there was 
confidence that the model was indeed of the highest quality possible. Our quality approach has been 
further validated with numerous Exemplary Course Awards from Blackboard, including: ENGL 221 
Technical Report Writing (2008), MBAA 514 Strategic Marketing Management in Aviation (2011), DAV 
733 Globalization and the Aviation Environment (2011), ENGL 222 Business Communication (2012), 
DAV 714 The Legal Environment of Aviation (2012), DAV 712 Aviation Safety Management Systems 
(2012), and DAV 713 The Economic Environment of Aviation (2012). 

Whatever the quality guidelines are, the way they are implemented and the communication that occurs 
around them are both critical elements to gaining acceptance from the stakeholder community. Study 
results regarding helpfulness of quality guidelines found the following approaches to be positive in 
gaining broad institutional acceptance of quality guidelines: (1) review guidelines at beginning and keep 
them in mind throughout the process; (2) use them as a checklist at the end of development; and (3) adapt 
to the needs of each course [17]. At Worldwide, the approach has consistently yielded positive results. 

A. Six Quality Standards for Course Content 

Based upon a critical review and compilation of evaluation criteria from numerous reputable sources, 
including Quality Matters, Blackboard, Southern Regional Education Board, and Southern Association of 
Colleges and Schools, the accrediting body for Embry-Riddle, 1DD holds the following standards as the 
basis of all course design requirements: 

Standard 1 - Basic Design: Course navigation, organization, and statements of expectations have clarity 
and alignment; style guidelines are adhered to systematically throughout the course. 

Standard 2 - Learning Objectives: Objectives are formulated as a subset of course and program 
outcomes and express measurable expectations student assessment. 

Standard 3 - Interactive Learning: Course design supports interactive learning essential for student 
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motivation, intellectual commitment, and personal development. 

Standard 4 - Instructional Materials: Comprehensive, current, and accurate instructional materials 
align with course outcomes and are prepared by qualified persons. 

Standard 5 - Course Technology: Course technology supports interactive learning and provides fully 
accessible modes of delivery, resources, and student support for its use. 

Standard 6 - Learning Assessments: Formative and summative assessments align with objectives, 
content, and online learning context [22-23]. 

Each standard has an accompanying, comprehensive subset of course design objectives. Many of the 
objective requirements have been integrated into the master template that is the foundation for each 
course at Worldwide. 

B. Master Template 

Research in online learning has consistently shown that students learn faster and with greater 
effectiveness when the framework in which the instruction is provided (also known as the online course 
“shell” at Worldwide) is predictable, consistent, and accurate [11] [15]. Using a standardized template 
greatly increases each of these values. In addition, the use of course templates streamlines the course 
production process [11]. Course templates also support quality assurance by setting various standards 
such as those related to navigation and structure, style, and instructional requirements [24]. 

The current online course instructional design model at Worldwide originated in 2007, with the 
introduction of such a master template, nicknamed the “Gold” template. This template increased 
consistency in course structure, navigation, and function. Template structure included standard, set areas 
of the course including “Start Here” (home to the course syllabi, instructor profile, student policies, online 
learning orientation, accessibility information and disability support), “Announcements”, “Modules”, 
“Resources” (course-specific an academic-general), “Discussion Board”, “Email”, “Help”, “My Grades”, 
and more. Each section contains specific elements designed to make learning easier through logical 
content and support resource organization and flow. Navigation refers to the order in which these sections 
are listed on the course menu, and the order in which elements in each of the sections are set. 
Standardized functions include specific tasks all students must complete, tools used commonly for 
specific activities, rules regarding feedback, and so on. Remaining course design objectives not yet met 
via the master template, are addressed through alignment procedures, formatting standards, and activity 
and assessment design, which are addressed next. 

C. Alignment 

Alignment is a critical component to achieving quality in online learning, or any educational program 
(See Figure 1. below) [25]. The alignment component is so critical, that it normally represents one of the 
major benchmarks for accreditation reviews [23]. In order to achieve program outcomes, there needs to be 
course-level outcomes accompanied with subsets of learning objectives dispersed into consistently 
structured units of learning. In Worldwide online courses, these are referred to as “modules.” Finally, 
learning objectives are individually mapped to unit-level activities (content presentation, practice, and 
assessment activities), containing both graded and ungraded tasks, through which students either work 
towards competency or demonstrate actual competency of the learning objectives. 

Program Outcomes Course Outcomes Module Learning Objectives Activities 

Alignment 


Figure 1. Alignment of Critical Course Components 

As with course design standards guidelines, the process of creating and documenting alignment must be 
flexible and adaptable to both the course developer needs and course subject matter requirements. Each 
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production coordinator has this flexibility concerning how the alignment process occurs, with careful 
consideration as to the developer’s needs and the requirements of the subject matter. Commonly used 
tools in achieving alignment include: 

Course Guide: Academic department-level document for every course in the Worldwide system, this 
document provides the learning outcomes required for each course, regardless of whether the course in 
online, on-ground, or blended. 

Outcomes and Objectives Planning and Alignment Tracking Matrix: Using Bloom’s Taxonomy as a 
guideline, course developers plan subsets of learning objectives for each course outcome, and then 
tentatively map them to modules in concurrence with the completion of the Preliminary Course 
Map/Schedule Planning document below. As course development proceeds, changes in the assignment of 
learning objectives to modules are reflected in the matrix, ensuring an accurate alignment map at the 
completion of the course production process [26], 

Preliminary Course Map/Schedule Planner: A course developer compiles all learning objectives in this 
high-level planning document that allows the developer to rough sketch out each module, including the 
module titles and the activities that will be designed to meet the learning objectives. 

Individual Module Template: Once the two planning documents above are approved by the academic 
department chair, then the module titles, learning objectives and activity titles are transferred over into an 
individual module template. Course developers then flesh out each activity. The production coordinator 
emphasizes in the course production kick-off meeting that the developers need not concern themselves 
with providing the “directional text” explaining to students where to go and what to do, from a course 
navigational/functional perspective, but rather simply to provide the content and specific activity task 
descriptions. The production coordinator will provide all directional text, much of it standardized. Finally, 
developers provide any grading guidelines/rubrics and assess the time-on-task for each activity, providing 
a time total for each module. Undergraduate courses have a different time-on-task (replacing traditional 
so-called “contact hours”) requirement (typically 5-8 hours per module) than graduate courses (typically 
8-12 hours per module). 

D. Style 

Each production coordinator in the development of the course content follows basic course formatting 
requirements in the Blackboard learning management system. There are three categories: 

Text Style: Specific font sizes, colors, and types are provided for headings and body content 

Web Design: Alternate tags, file formats, file sizes, hyperlink, image, embedded document construction 

and behaviors, etc. 

Directional Text: Standard navigation and functionality instructions for students to complete tasks using 
the built-in learning management system tools 

The 1DD team has been careful (and through some trial and error) not to overextend the style 
requirements, making them too complex, constraining or overly taxing. There is room for each individual 
production coordinator to be creative and innovative, within reason. 

E. Checks and Balances: Course Development Reviews 

For 1DD, the process of critical self-examination as a path to QA is manifested through a system of 
checks and balances called the reviews process. Reviews are typically tied to milestones in the production 
process, as illustrated in Figure 2 below in the paper and their related stages of the process. Course 
development reviews are conducted according to the following schedule, in the order shown below. The 
process appears strenuous, but once the first two reviews are completed and revisions made to satisfaction 
of all involved production team members, generally, the remaining reviews are quite smooth. Depending 
on the delivery timeframe and other factors, this process can be flexible, with some reviews happening 
simultaneously in cases where time is tight. 
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F. Planning Documents Reviews (Stage 3/Milestone 2) 

Lead Instructional Designer Review: Checks for alignment, wording of learning objectives, may make 
activity suggestions to meet learning objective needs 

Academic (Department Chair) Review: Checks and approves planning documents, ensures alignment 
of proposed learning objectives and activities/assessments with course outcomes and program outcomes. 

Modules 1 and 2 Reviews (Stage 4/Milestone 3) 

CD Review: Checks content of Modules 1 and 2 

Lead Instructional Designer Review: Checks primarily for alignment and adherence to style guidelines 
and time-on-task per module; may also recommend some activity/assessment/evaluation (rubric) setup 
changes at this point 

Academic (Department Chair) Review: Checks and approves overall structure of course this far, 
including alignment, learning objectives, activities and assessments 

Module 3-6 Reviews (Stage 4/Milestone 4) 

CD Review: Checks content of Modules 3-6 

Final Reviews (inclusive of Modules 7-9/12) (Stages 5 and 6/Milestones 5 and 6) 

CD Review: Checks content of Modules 7-9/12, Discussion, Exams, and Resources areas, Grade Center, 
plus additional documentation (syllabus, instructor guidance, etc.) 

Peer Production Review: Checks for technical issues (hyperlinks) and spelling/grammatical errors 
Lead Instructional Designer Review: Checks primarily for alignment and adherence to style guidelines 
and time-on-task per module 

Director Review: Does final check; updates any new or newly-revised template items; checks Grade 
Center setup 

Academic (Department Chair) Review: Checks and approves overall structure of course, including 
alignment, learning objectives, activities and assessments 

Depending on the nature and complexity, reviews may also be part of post-production tasks, such as 
initial delivery and pilot course revision and regular maintenance and updates. 

The concept of an evaluative review, in which criticism is sometimes constructive and sometimes not, 
will find strong resistance in any bastion of autonomy such as a post-secondary educational institution. 
However, it also appears there are ways to mitigate resistance and use reviews to maximize potential for 
quality, unlike any other tools available in instructional design arsenal. Some of the review guidelines that 
have been found to decrease resistance and therefore increase the success of reviews being accepted and 
useful include: 

Begin with a review of the course syllabus and any instructor guidance documentation to gain an 
overview of the course, then refer back to them regularly to make sure that the content in the course 
matches those documents. 

Chart review comments in table format and provide a clear pathway to the location under comment. 

The reviewer should not correct any mistakes during a review; this could be a mistake that is duplicated in 
other areas of the course with which only the production coordinator might be familiar. 

Do not use superlatives or derogatory remarks, for example, "this is the most confusing rubric 1 have ever 
seen" or "1 have seen better composition from my 6th grader." 

The course design and production process at Worldwide is aimed at producing turnkey style courses to be 
taught by a highly qualified pool of over 800 instructors. Given the high number of online courses and 
tremendous number of live sections running at any given time, maintaining quality was a significant 
concern. The model of faculty instructors each producing and delivering their own online course would 
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make achieving consistency in design and delivery difficult. While this production model is common at 
many schools, it is known to be ineffective so at Worldwide it was decided to centralize the process using 
a collaborative course production team, facilitated through 1DD department [27-28], Over time, a process 
evolved that ensures healthy collaboration among production team members and meeting quality 
standards based on sound learning, teaching and instructional design theoretical foundations, both factors 
cited as key influences on the success of distance learning efforts [17]. 

V. HISTORY AND CHALLENGE 

Online learning began at Worldwide in 1993 as part of a hybrid program that combined video and a 
homegrown online bulletin board system for interaction. WebCT was the first fully online learning 
management system, adopted in the late 1990’s, followed by Blackboard in 2000. As mainstream online 
learning grew rapidly in popularity, there was tremendous concern for the quality of the educational 
experience [9] [10][14 ][29]. For Worldwide, the challenge was to produce, deliver, and maintain online 
courses with the highest quality possible to thousands of primarily non-traditional students all over the 
world. Worldwide has not been alone in coping with this problem. Higher education institutions typically 
do not have the resources needed to produce online learning to meet demand efficiently and effectively 
[25] [30]. Online learning development frequently suffers from a lack of resources, particularly 
infrastructure, policy and support mechanisms, and is conducted under pressure to quickly meet growing 
demand. As a result, student attrition rates are high and faculty blame the administration for failure to 
provide adequate resources. In addition, faculty are subject matter experts, but not always instructional 
design experts, and having a lack of instructional design expertise, especially specific to online learning, 
is seen as a significant cause of failure in an online learning program. Unfortunately, many educational 
institutions still follow this model of faculty as subject matter expert, instructional designer, content 
producer, learning management system developer, and student support, from admissions and technology 
troubleshooting [11] [31-33]. This is sometimes referred to as the “lone wolf’ or the “lone ranger” model 
of course production and delivery [17] [28], This approach has proven to be not scalable and does not 
lend itself to the diffusion of innovative practice in an organization [9]. Adding to the pressure, by 2005, 
quality assurance standards for online learning had been implemented by most accrediting bodies and 
many of the other significant educational organizations that monitor and evaluate educational providers, 
from Embry-Riddle’s accrediting agency, the Southern Association of Colleges and School to the United 
Nations Educational, Scientific and Cultural Organization [23]. 

VI. SOLUTION 

Worldwide recognized, and wisely so, that the “lone wolf’ approach was not going to be sufficient for 
their model of delivery and that upfront investment in sufficient resources would be required to convert to 
a centralized approach [28]. Restauri contended that a second model, one that creates this centralization, 
utilizes a collaborative team approach to instructional design, and is much more effective and efficient at 
producing quality in online learning and receives better faculty buy-in once proven [34]. Instead of the 
faculty member performing all the roles above, a division of labor is established in which experts trained 
in each specialty come together and apply best practices and a sound instructional design process to 
collaboratively produce and maintain online courses, all in an environment that effectively supports the 
information technology infrastructure [11] [15] [17] [31-33] [35]. Thi s was referred to by Laird as the 
integration model: all institutional resources cooperate to produce online learning aligned with the 
organization’s mission and goals [25]. Research has shown this centralized model to be the most cost- 
efficient while producing the highest quality courses in which faculty can satisfactorily focus on teaching 
[11] [36]. This approach has been used on a large scale with success at other universities, such as Royal 
Roads University in British Columbia, Canada [17]. A similar centralized solution, adopted in 2002 in its 
earliest rendition at Worldwide, came in a multifaceted form involving several key components: a 
collaborative production team with clearly identified division of labor; a flexible instructional design 
process; a master template and course shells management strategy; a system of checks and balances 
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created through a highly constructive reviews process; and responsive maintenance, all with quality at the 
heart of the model. 


VII. COURSE PRODUCTION PROCEDURES 


The Worldwide course production process model has evolved and refined over time and currently is 
working very well for departmental needs, given the constraints and environmental conditions, to meet 
production demand. New course productions follow a process similar to the flowchart below. 


LAdditional course documents; Syllabus / Instructor info / Developer Welcome 




Planning Module land 2 

Modules 3-6 

document:! Templates and 

Templates 

Textbook Bb Input 


Adoption 

Input 


Planning document: Course map 


Planning document; Alignment 




Modules 7-12 
Templates and Bb 
Input 



Time 


Milestone (MS) Events 



Kick-Off Meeting 



Planning Documentation 
Re v i e ws/Ap p r o va I 



Moduel 1 and 2 
Reviews/Approval 


Modules 3-6 Reviews/Approval 


Modules 7-12 Reviews/Approve 


Course Delivery 


Figure 2. Embry-Riddle Worldwide Course Production Process Model with Detail 
While the intent here is to not propose a new instructional design model, the model above is intended to 
only illustrate the production process used at Worldwide as accurately as possible. This representation 
gives a snapshot of our current instructional design process and is modeled upon existing, accepted 
instructional design theoretical frameworks. While it is primarily designed based loosely upon the Rapid 
Prototyping model, the Worldwide model draws from best practice elements of many instructional design 
models, both linear and non-linear [37]. The semi-linear, overlapping process illustrated above may also 
be organized into stages, providing another perspective of the process: 

Stage 1: Course production prep, CD contract, and materials (textbook, case studies, etc.) adoption 

Stage 2: Course production kick-off meeting 

Stage 3: Planning documents completion and approvals 

Stage 4: Module, media, and assessment development in Blackboard with reviews at intervals 

Stage 5: Additional course documents (syllabus, rubrics, instructor memo, etc.) and final developer 
review 

Stage 6: Final production peer review 

Stage 7: Final academic review and course production delivery 

How much time does the course production process take? Ideally, given that any one production 
coordinator is working on multiple projects simultaneously, the ideal time frame for a new Gold course 
production is no less than four months. Developers, too, have regular teaching responsibilities and other 
non-course production professional activities to attend to, and therefore they must also fit their course 
production duties into their hectic schedules. 

In reality, a full 4-month timeframe is not always possible with production coordinators having been 
given new course developments with deadline timeframe as little as two months. The following chart 
illustrates time requirements, organized by hours per task category, for a single, typical course production. 
Keep in mind that many factors affect these time requirements, from content development delays, 
administrative delays, review delays, technology upgrades, etc. This data comes from a record of a single 
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course production, AVS 1000 Private Pilot Online Ground School, produced between August and 
October of 2008. The total production time equaled approximately 260 hours, which is about 6 V 2 weeks 
of full-time work. 


VIII. ADOPTING EMERGING TECHNOLOGIES IN THE PRODUCTION 

PROCESS 


The use of Web 2.0 communication tools and cloud applications forms much of the basis of the 
collaboration among production team members, and at every stage of the production process. While the 
planning documents and individual module templates have traditionally been developed using a word¬ 
processing program, and email has and still serves as the primary communication tool for the ongoing 
exchange of simple messages, much of the “backstage” work on course development is now being 
facilitated through online tools such as: Doodle (meeting scheduling); JoinMe or ERAU’s own Saba 
Centra EagleVision (Web conferencing); Google Docs, Drive, and Sites (Worldwide Textbook 
Forecasting Spreadsheet, 1DD Production Schedule, and 1DD Production Team Website); Microsoft’s 
OneNote and SkyDrive (Individual Module Templates); PB Wiki (Course Maintenance Wiki); and 
Blogger (1DD Production Blog “Resources for the Instructional Design and Development of Learning 
Environments” or “RIDDLE”). Experimentation with new tools is encouraged to enhance collaboration 
and increase efficiency, but the tolerance of the course developer in accepting newer technologies for 
completing their tasks is always considered, and typically the production coordinator will use whichever 
tool(s) the developer is most comfortable with. On the far, cutting edge, Second Life is being 
experimented with for learning and teaching, but is primarily used at this time for the development of 
videos and other multimedia for use in courses. The Worldwide Island in Second Life is designed as a 
virtual airport, with the intention of one day using it for airport operations simulations (See Figure 3 
below). 



Figure 3. Embry-Riddle-Worldwide Island in Second Life. 


IX. SUMMARY OF BEST PRACTICES IN COURSE DESIGN AND 

DEVELOPMENT 


No matter the size of the institution, the following course design best practices may be followed 
regardless of budgetary or other physical resource constraints that may exist. If anything, organizational 
culture may be the biggest hurdle to overcome, but with diplomacy, consensus building, and time, these 
strategies are accomplishable. 

Set a baseline of standardization achieved via a master course template and promote course design 
standards across the institution. 
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Focus on developing healthy, functional production team relationships; prevent power struggles at all 
costs. Often this requires the instructional designer to serve in a somewhat subservient role, while 
continuing to provide their expertise in a gentle way. If a production team is not working, adjust the 
members of the team as needed. Some pairings simply do not have the right chemistry, even though the 
individuals may work wonderfully with others. 

Plan for course quality. Alignment via the development of sound learning objectives should be the main 
aim of planning, followed closely by creating varied interaction points throughout a course structure, 
developing a variety of assessment tools, and evaluating time/effort-on-task for activities, adjusting 
appropriately for the level of the course. 

Enact a system of checks and balances: use reviews at multiple stages in the course production process. If 
reviews are not already part of the process, then provide evidence for why reviews are critical and form 
an advisory committee of a cross-section of stakeholders to identify a path of least-resistance (and there 
will always be resistance) to achieving a system. Ensure that reviews are constructive and respectful by 
providing guidelines to achieving such. 

Provide instruments for easy and timely student and instructor feedback to encourage technical 
maintenance and subject matter accuracy and track maintenance work. 

X. FACULTY DEVELOPMENT 

In 2007, a need for Faculty Development was identified and a series of courses was created on the course 
design template. The courses, taught by OFQMs, certified current and new instructors to teach the online 
courses at Worldwide. The series won a Program of Excellence-Non-Credit division award from UCEA. 

In 2010, the series was franchised outside the online campus to serve the 150 global campus locations and 
two residential campuses. Previous students became instructors for the Eastern, Central, Western, and 
International regions to certify the instructors teaching at the campuses located in the regions. In 2011, the 
Rothwell Center for Teaching and Learning Excellence began guiding the University-wide initiative to 
promote the cross-utilization of instructional resources in support of the Five Ways to Learn marketing 
strategy. 

A. Finding and Qualifying Online Instructors 

A variety of marketing methods are used to locate potential instructors by the Department of Online 
Instruction. Print advertisements in academic publications and trade journals are costly but generally 
effective. Still practical, personal and social networks of current faculty, alumni, and industry 
professionals promote the online teaching opportunities at Worldwide. In the Credentialing process, 
instructors are interviewed by the OFQMs for their discipline before proceeding to Faculty Development 
training. 

During the Credentialing process, potential instructors are required to attend the faculty development 
courses. The courses certify they are prepared to teach online courses. If instructors teaching at a 
residential campus want to include an online course in their assignment list, they also must become 
certified through the Faculty Development courses to teach online. 

B. Faculty Development Series 

The Rothwell Center for Teaching and Learning Excellence maintains the faculty development series and 
registers the potential faculty as students. The courses are 4 weeks; instructor facilitated and designed to 
give faculty an immersed student experience within the online course template. 

The important question when developing a program for faculty is, what does the faculty need to know to 
teach an online course? The question directed the development of the course learning objectives and 
identified the desired competencies a student would achieve upon completion of the faculty development 
series. There is an expectation the student will obtain the “skills, attitudes, and abilities in a given context 
that adjust and develop with time and needs in order to effectively and efficiently accomplish a task and 
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that are measured against a minimum standard [38].” The Quality Managers who taught the courses also 
found the series to be a good indicator of a student’s future performance when the student became the 
instructor in their course. 

1. FACD - Instructional Use of Blackboard and Lab. 

The course was designed to provide a hands-on experience to develop basic navigation skills using the 
online Blackboard tools and to become familiar with a module course design. The course contains a lab to 
construct samples. In the lab environment the student role transforms into an instructor role. After grading 
students can practice, revise and save samples to use in their course. 

Prior knowledge of the Blackboard tools can be helpful or a serious problem. The FACD instructor must 
impress on the student the importance of how the tools will be used in the online course. When the 
student becomes an online instructor they are restricted by the quality management of instructional 
expectations, the student’s learning experience, documenting the learner’s activities, maintaining the pre¬ 
designed course content, and University values. Their focus will not be centered on content and design 
considerations, but on teaching their respective subject matter. 

2. FACD - Teaching at ERAU-Worldwide. 

Enrolled concurrently with the FACD Instructional Use of Blackboard course, students practice active 
learning by modeling student-to-student and student-to-instructor interaction in threaded discussion 
forums. Without using a textbook, the instructional content of all the courses relies on readings, activities, 
and multimedia resources to complement the discussions. Students are encouraged to reflect on the role of 
the student and instructor, online teaching theories, and the dynamics formed in teaching and learning 
online in their written assignments. Of the learning theories included, importance of instructor contact 
with the students to promote active learning is stressed [39]. A highlight of the course is an assignment 
that compares and contrasts the student’s instructional philosophy to their learning experiences in a 
reflective essay. The course was also designed to be used as a stand-alone orientation for new personnel at 
the branch campuses. 

3. FACD - Collaborate Using Blackboard Tools. 

Continuing to model the standard course design template, the focus is on collaboration, working in a 
group, and using social media tools. The course increases practice with the tools and interacting in the 
discussion forums. The written assignments range from reviewing instructional online theory, researching 
best practice examples, and reflecting on their instructional experiences and new online skills. 

In discussions, students may share instructional resources they have created and used successfully in their 
courses. The instructor should describe the limitations that may be imposed in their use of their materials 
in their courses. Content ownership can be very important to instructors, but far more important will be 
their attitude if they take a defensive position with student unhappy with the course design and 
assignments [40]. 

For many students, the course holds several first time experiences; an opportunity to share responsibilities 
in creating instructional content, construct a blog or wiki, and work on a group project. All projects are 
opened for review and critique. Topics are determined by the group members after some research, 
learning what the differences are, and keeping in mind of the project must be created in a limited amount 
of time. The most popular topics are developed as course resources to assist students in communication 
skills, plagiarism, and APA citation. Students having very good technology skills or experience in 
creating a blog or wiki have included animations, color, and graphics to illustrate their project. 

C. Qualified and Competent Faculty 

What was once a faculty development series of courses specific to developing online faculty 
competencies has become a university-wide program? The program increases the mobility of the full-time 
and adjunct faculty to seamlessly teach in different learning modalities. 
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All faculty become qualified to a defined standard of use of the technology tools, teaching, and learning 
competency. They test the template design model and develop the desired attitudes and abilities of 
teaching through the interaction of the student and instructor experience. The courses are taught by 
certified faculty from a variety of teaching disciplines and in locations around the world. 

1. Online Instruction Policies 

New instructors are exposed to the department’s policies during faculty development and the orientation 
course. A number of documents are housed within the courses, as well as in the Instructor Resources area 
of the Worldwide Online Faculty organization. These publications, authored and maintained by the 
OFQMs, include the Online Faculty Manual, the Online Instructor Checklist, and the Expectations and 
Obligations of Faculty memo. Compliance with direction in these documents is compulsory, per the 
Instructor Contract. 

The Online Faculty Manual was originally designed to answer the most frequently asked questions from 
faculty and to cut down on the number of calls with common questions and needs. It has become the go-to 
document for most any question about online operations for faculty members, addressing all aspects of 
the course from setup to final grades, and more. An Administrative Course Information section covers 
course access, modes of Blackboard, course setup tasks, integrity violations, and the role of the faculty 
course developer. Accessing the Blackboard course via Safari in iPads and iPhones is also defined with 
step-by-step instructions. The manual also spells out student administrative information, such as drop/add 
periods, student withdrawals, what to do about inactive students, conflict resolution, student 
responsibilities, and disability support information. A refresher on the Family Education Rights and 
Privacy Act (FERPA) of 1974 is also included. 

Basic Blackboard operations are also addressed. Course navigation, location of materials, posting a 
faculty profile, discussion board operations, and posting announcements are all pre-term actions that 
sometimes require a memory jogger. Making exams available, emailing form within the course, and 
grading in the Grade Center also drew many questions before the Online Faculty Manual became all 
encompassing. More complicated course tools, such as SafeAssign, course reports, and group 
collaboration, also get several pages dedicated to them in the manual 

The Online Instructor Checklist, mentioned previously, is an all-encompassing 6-page document, which 
steps the instructor through every aspect of teaching a course for Worldwide. Most steps include enough 
detail that even new instructors can function without handholding during their first teaching experience. 

Finally, the Expectations and Obligations memo lays the ground rules for instructors, to include 
attendance requirements, expectations regarding student interaction, and other contractual obligations. 
Compliance with this document is compulsory. The OFQMs review all these documents regularly to 
ensure that all information contained therein is still timely and applicable. 

2. Online Faculty Quality Management 

The OFQMs monitor faculty members throughout each term, ensuring the continued outstanding 
instructional quality of online courses from Worldwide. Initially, the OFQM position was more of a 
minor administrative/support function within the department—answering instructor questions, addressing 
student complaints. Overtime, the OFQM has evolved into more of a management-coaching role. OFQM 
are peer faculty, often within the discipline of the instructors they manage. In some cases, the OFQM 
actually teaches courses within the department, adding practical experience and ensuring familiarity with 
courses, as well as some of the discipline-specific issues instructors might experience. OFQMs guide and 
assist faculty members during their teaching assignments by ensuring proper course setup, monitoring 
performance and adherence to policies, and are available to assist faculty with any number of issues. 

While plagiarism and conflict resolution are discussed during orientation, it is not uncommon for an 
instructor to ask for assistance when one of these situations actually occurs during a term. For example, 
instructors are often hesitant to level a plagiarism charge for the first time. Clarification of polices and a 
more in-depth discussion of SafeAssign is always requested when an instructor first encounters such a 
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situation. OFQMs are versed at procedures, and with years of experience, can offer insight into the 
SafeAssign originality reports and advise the instructor as to the difference between a real issue or a 
“false positive,” which on the outside looks like a serious case, but after further investigation, turns out 
not to be an issue at all. 

OFQMs also review each instructor’s performance during the live term as often as possible, and provide a 
formal Course Review, detailing the instructor’s attendance, participation in activities, timeliness of 
response, grade center maintenance, course setup requirements, and other information. The quality of 
instruction within the course is of great importance, with both the number of instructor responses and 
quality of involvement being evaluated. Detailed feedback is given on the instructional course review, 
with special attention paid to items deemed unsatisfactory by the Quality Manager. Coaching after an 
unsatisfactory review is common. 

Course Reviews are also generated by the Worldwide Online Query system. The WO Query is the 
department’s standardized intake mechanism for student questions and complaints that have not been 
solved by the student’s contact with the instructor. This form, located in a common area in Blackboard, is 
filled out by a staff member on behalf of the student, and is addressed by the OFQM. Simple needs, such 
as missing grades, or unavailable exams, can often be handled quickly, but the occasional query alerts the 
OFQM to an issue that must be addressed directly with the instructor. Ineffective instructional 
performance rarely goes unnoticed by students. Many times, the student’s complaint is unfounded, but 
careful review and investigation are required to be sure. OFQMs investigate the issue fully, and if 
necessary, perform a full course review, including documentation on the course review form. Again, 
coaching might be necessary to correct instructor behavior within the online classroom in areas such as 
student interaction, timeliness of feedback or grading, quality of feedback, and other issues that 
commonly turn up. Resolving the query effectively and in a timely manner is the goal, but the occasional 
query sometimes takes days to resolve. 

Occasionally, an instructor question or a student query will bring to light an issue with the course itself. 
OFQM often consult with the faculty course developers and the production coordinators in IDD who 
build the online courses on content and design issues. Features in Blackboard, while often intuitive, do 
not always work correctly. Moving course tools that are typically located within the course menu to a new 
location sometimes causes confusion for instructors and students. Quality Managers relay instructor 
feedback to IDD personnel as necessary in an effort to include as much good information and as many 
great ideas as possible, ensuring the best possible courses for students and faculty. 

The OFQM’s role is important in ensuring our students get the course experience they expected when 
they enrolled in a Worldwide online course. While all of the Quality Managers perform relatively the 
same function, as a team, they are matchless body of knowledge of experience. Working together, they 
ensure all currently teaching faculty are provided the same up-to-date information each term 

XI. SUMMARY: LESSONS LEARNED 

To close this article, a summary of conclusions or lessons learned from this inquiry are presented here. 
Each of the significant areas of the inquiry are considered concisely below. 

A. Literature 

Quality assurance is a core value, and underlies many policy decisions in a higher education organization 
and are directly influenced by numerous situational and environmental factors, including accreditation, 
technology, and competitiveness. Support for and views with regards to how online learning should be 
administered, particularly with the course design component, are diverse several authors have shown that 
accreditation as the single most significant factor that rises above the others and pushes institutions to 
seek ways to increase quality. This push emanates to all areas of the instructional design and deployment 
process as discussed within the paper. 

B. Course Design and Production 


32 


Journal of Asynchronous Learning Networks, Volume 16: Issue 5 



Large Scale Quality Engineering in Distance Learning Programs 


The production model is common at many schools, and is seen to be ineffective so Worldwide centralized 
the process using a collaborative course production team, facilitated through 1DD department. A process 
evolved that ensures healthy collaboration among production team members and attainment of quality 
standards based on sound learning, teaching and instructional design theoretical foundations, both factors 
cited as key influences on the success of distance learning efforts. The process employs standardization 
via a master course template and promotes course design standards across the institution yet it emphasizes 
the working relationships among development personnel. There is a system of checks and balances 
through multiple stages reviews, which are aimed at being constructive. Students and course instructors 
are provided instruments for easy and timely feedback to encourage technical maintenance and subject 
matter accuracy while also tracking maintenance work. 

C. Faculty 

From the recruitment, faculty development, teaching and course development processes that involve the 
instructor within the educational process, it is clear that their work cannot be done effectively without the 
inclusion of personnel providing support from 1DD or Online Instruction. It is as a team that these 
individuals ensure during the lifecycles of production and instruction, the highest quality product within 
the classroom and/or online environment. 
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